Object identification apparatus, object identification method, and program

ABSTRACT

An input image showing a same object as an object shown in a reference image is identified more accurately. A difference area in the input image is determined by converting a difference area in the reference image, on a basis of geometric transformation information calculated by an analysis using a local descriptor. By matching a descriptor extracted from the difference area in the input image with the difference area in the reference image, fine differences that cannot be identified by conventional matching using only a local descriptor can be distinguished and images showing a same object can be exclusively identified.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage application of InternationalApplication No. PCT/JP2013/063995 entitled “Object IdentificationApparatus, Object Identification Method, and Program,” filed on May 21,2013, which claims the benefit of the priority of Japanese patentapplication No. 2012-184534, filed on Aug. 23, 2012, the disclosures ofeach of which are hereby incorporated by reference in their entirety.

BACKGROUND

The present invention relates to an apparatus, a method, and a programfor accurately identifying an object in an image.

To enable robust identification of a subject in an image with respect tovariations in photographed size and angle and to occlusion, systems areproposed which detect a large number of characteristic points (featurepoints) in the image and which extract a descriptor of a local area (alocal descriptor) around each feature point. As representative systemsthereof, Patent Document 1 and Non-Patent Document 1 disclose localdescriptor extraction apparatuses that use a SIFT (Scale InvariantFeature Transform) descriptor.

Conventionally, with a local descriptor extraction apparatus,information related to brightness is first exclusively extracted fromeach pixel in an image, a large number of characteristic points (featurepoints) are detected from the extracted brightness information, andfeature point information that is information related to each featurepoint is outputted. In this case, feature point information indicates,for example, a coordinate position or a scale of a detected localfeature point or an orientation of a feature point. Subsequently, alocal area from which descriptor extraction is to be performed isacquired from the feature point information that is a coordinateposition, a scale, an orientation, or the like of each detected featurepoint to generate (describe) a local descriptor.

For example, as described in Non-Patent Document 1, in order to identifyan image showing a same subject as a subject in a photographed image, alocal descriptor 1 extracted from the photographed image or, in otherwords, an input image is compared with a local descriptor 2 generatedfrom a reference image. Specifically, distance calculations on a featurespace are performed on all combinations of respective descriptors ofareas in a vicinity of feature points constituting the local descriptor1 and respective descriptors of areas in a vicinity of feature pointsconstituting the local descriptor 2. A nearest descriptor is determinedas a corresponding descriptor. The corresponding descriptor isdetermined so as to also correspond to a feature point that is a sourceof descriptor generation. Subsequently, regarding a combination offeature points determined to be corresponding feature points, whetherthe corresponding feature points are correct or erroneous is determinedbased on whether or not coordinate positions resulting from movingcoordinate positions of the feature points in the input image inaccordance with a specific geometric transformation are consistent withcoordinate positions of the feature points in the reference image. Whenthe number of feature points determined to be correctly correspondingfeature points is equal to or larger than a prescribed value, it isdetermined that a same subject is shown (in other words, the subject inthe input image and the subject in the reference image are consistentwith each other).

-   Patent Document 1: U.S. Pat. No. 6,711,293-   Patent Document 2: Patent Publication JP2010-79545A-   Non-Patent Document 1: David G. Lowe, “Distinctive image features    from scale-invariant keypoints”, USA, International Journal of    Computer Vision, 60 (2), 2004, pages. 91-110

Conventional object identification systems that utilize localdescriptors identify an object based on a correspondence relationshipbetween a local descriptor extracted from brightness information of aninput image and a local descriptor extracted from brightness informationof a reference image. With such an identification method, when an objectshown in the input image and an object shown in the reference imagediffer from each other but the difference between the two objects isminute, there is a problem that the images are erroneously identified toshow a same object due to the existence of a large number ofcorresponding feature points.

SUMMARY

The present invention has been made in consideration of the problemdescribed above and an object thereof is to provide a technique for moreaccurately identifying an image showing a same object as an object shownin another image.

An object identification apparatus according to the present inventionincludes: a local descriptor matching unit for determining whether ornot respective descriptors of feature points extracted from an inputimage and respective descriptors of feature points extracted from areference image correctly correspond to one another; an input imagedifference area descriptor extracting unit for extracting a descriptorof an area in the input image corresponding to a position of an imagearea obtained by performing a geometric transformation for correcting ageometric deviation between the input image and the reference image on aprescribed area of the reference image when a score based on the numberof combinations of descriptors determined to correspond correctly by thelocal descriptor matching unit is equal to or larger than a prescribedvalue; and a descriptor matching unit for matching the descriptorextracted by the input image difference area descriptor extracting unitwith a descriptor extracted from the prescribed area of the referenceimage, and for outputting a matching result.

An object identification method according to the present inventionincludes: a local descriptor matching step of determining whether or notrespective descriptors of feature points extracted from an input imageand respective descriptors of feature points extracted from a referenceimage correctly correspond to one another; an input image differencearea descriptor extracting step of extracting a descriptor of an area inthe input image corresponding to a position of an image area obtained byperforming a geometric transformation for correcting a geometricdeviation between the input image and the reference image on aprescribed area of the reference image when a score based on the numberof combinations of descriptors determined to correspond correctly in thedetermining step is equal to or larger than a prescribed value; and adescriptor matching step of matching the descriptor extracted in theextracting step with a descriptor extracted from the prescribed area ofthe reference image, and outputting a matching result.

A program according to the present invention causes a computer tofunction as: a local descriptor matching unit for determining whether ornot respective descriptors of feature points extracted from an inputimage and respective descriptors of feature points extracted from areference image correctly correspond to one another; input imagedifference area descriptor extracting for extracting a descriptor of anarea in the input image corresponding to a position of an image areaobtained by performing a geometric transformation for correcting ageometric deviation between the input image and the reference image on aprescribed area of the reference image when a score based on the numberof combinations of descriptors determined to correspond correctly by thelocal descriptor matching unit is equal to or larger than a prescribedvalue; and a descriptor matching unit for matching the descriptorextracted by the input image difference area descriptor extracting unitwith a descriptor extracted from the prescribed area of the referenceimage, and for outputting a matching result.

According to the present invention, a technique for more accuratelyidentifying an image showing a same object as an object shown in anotherimage can be provided.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram representing a configuration example of anobject identification apparatus according to a first embodiment;

FIG. 2 is a flow chart showing an operation example of an input imagedifference area determining unit 13;

FIG. 3 is a block diagram representing a configuration example of alocal descriptor extracting unit 11;

FIG. 4 is a block diagram representing a configuration example of alocal descriptor matching unit 12;

FIG. 5 is a block diagram representing a configuration example of aninput image difference area descriptor extracting unit 14;

FIG. 6 is a block diagram representing a configuration example of adescriptor matching unit 15;

FIG. 7 is a block diagram representing a configuration example of anobject identification apparatus according to a modification of the firstembodiment;

FIG. 8 is a conceptual diagram showing relationships of a differencearea with respect to a reference image and an input image differencearea with respect to an input image;

FIG. 9 is a conceptual diagram showing relationships of a differencearea with respect to a reference image and an input image differencearea with respect to an input image;

FIG. 10 is a conceptual diagram showing relationships of a differencearea with respect to a reference image and an input image differencearea with respect to an input image;

FIG. 11 is a conceptual diagram showing relationships of a differencearea with respect to a reference image and an input image differencearea with respect to an input image;

FIG. 12 is a block diagram representing a configuration example of anobject identification apparatus according to a second embodiment;

FIG. 13 is a block diagram representing a configuration example of alocal descriptor matching unit 16;

FIG. 14 is a block diagram representing a configuration example of adifference area estimating unit 17;

FIG. 15 is a block diagram representing a configuration example of adifference area descriptor extracting unit 18;

FIG. 16 is a block diagram representing a configuration example of anobject identification apparatus according to a third embodiment;

FIG. 17 is a block diagram representing a configuration example of adifference area estimating unit 19;

FIG. 18 is a block diagram representing a configuration example of anobject identification apparatus according to a fourth embodiment;

FIG. 19 is a block diagram representing a configuration example of adifference area estimating unit 20;

FIG. 20 is a block diagram representing a configuration example of anobject identification apparatus according to a fifth embodiment;

FIG. 21 is a block diagram representing a configuration example of adifference area estimating unit 21;

FIG. 22 is a block diagram representing a configuration example of thedifference area estimating unit 21;

FIG. 23 is a block diagram representing a configuration example of thedifference area estimating unit 21;

FIG. 24 is a block diagram representing a configuration example of thedifference area estimating unit 21;

FIG. 25 is a block diagram representing a configuration example of anobject identification apparatus according to a sixth embodiment;

FIG. 26 is a block diagram representing a configuration example of adifference area estimating unit 22;

FIG. 27 is a block diagram representing a configuration example of thedifference area estimating unit 22;

FIG. 28 is a block diagram representing a configuration example of anobject identification apparatus according to a seventh embodiment;

FIG. 29 is a block diagram representing a configuration example of adifference area estimating unit 23;

FIG. 30 is a block diagram representing a configuration example of thedifference area estimating unit 23;

FIG. 31 is a block diagram representing a configuration example of thedifference area estimating unit 23;

FIG. 32 is a block diagram representing a configuration example of anobject identification apparatus according to an eighth embodiment;

FIG. 33 is a block diagram representing a configuration example of adifference area estimating unit 24;

FIG. 34 is a block diagram representing a configuration example of thedifference area estimating unit 24;

FIG. 35 is a block diagram representing a configuration example of thedifference area estimating unit 24;

FIG. 36 is a block diagram representing a configuration example of anobject identification apparatus according to a ninth embodiment;

FIG. 37 is a block diagram representing a configuration example of adifference area estimating unit 25;

FIG. 38 is a block diagram representing a configuration example of thedifference area estimating unit 25;

FIG. 39 is a block diagram representing a configuration example of anobject identification apparatus according to a tenth embodiment;

FIG. 40 is a block diagram representing a configuration example of aninput image difference area descriptor extracting unit 26;

FIG. 41 is a block diagram representing a configuration example of adescriptor matching unit 27;

FIG. 42 is a block diagram representing a configuration example of anobject identification apparatus according to an eleventh embodiment;

FIG. 43 is a block diagram representing a configuration example of alocal descriptor matching unit 28;

FIG. 44 is a block diagram representing a configuration example of adescriptor matching unit 29; and

FIG. 45 is a block diagram representing a configuration example of anidentification score integration determining unit 30.

DETAILED DESCRIPTION First Embodiment

A first embodiment of the present invention will be described withreference to the drawings.

FIG. 1 is a block diagram showing a configuration of an objectidentification apparatus according to the first embodiment. The objectidentification apparatus includes a local descriptor extracting unit 11,a local descriptor matching unit 12, an input image difference areadetermining unit 13, an input image difference area descriptorextracting unit 14, and a descriptor matching unit 15. For example, theobject identification apparatus can be configured using an informationprocessing apparatus such as a personal computer or a mobile informationterminal. In addition, functions of respective units that constitute theobject identification apparatus are realized by having, for example, aprocessor expand a program stored in a storage area onto a memory andexecute the program. Moreover, components of other embodiments to bedescribed later can be realized in a similar manner.

The local descriptor extracting unit 11 detects a feature point from aninput image and extracts a descriptor of the detected feature point anda local area that is an area near the feature point as a localdescriptor. Details of a process performed by the local descriptorextracting unit 11 will be described later.

The local descriptor matching unit 12 matches a local descriptor 1extracted from the input image by the local descriptor extracting unit11 and a local descriptor 2 extracted from a reference image with eachother and identifies a corresponding local descriptor. Details of amethod for identifying a corresponding local descriptor will bedescribed later with reference to FIG. 4. The local descriptor matchingunit 12 identifies a corresponding local area between the input imageand the reference image in accordance with a position when a local areato which the local descriptor corresponds is geometrically converted.For example, when a coordinate position of a local area in the inputimage after being rotationally moved by a prescribed angle around acenter of the image is consistent with a coordinate position of a localarea to which the local descriptor corresponds in the reference image,the local areas with consistent coordinate positions in the input imageand the reference image are identified as corresponding local areas. Inother words, the geometric transformation described above is performedso as to correct a geometric deviation between the reference image andthe input image. In addition, when a corresponding local area isidentified, the local descriptor matching unit 12 outputs information onthe used geometric transformation (geometric transformation information)and a local feature identified image ID that is an image ID of thereference image to which the local area is determined to correspond.

The local descriptor 2 extracted from the reference image may beextracted from a plurality of reference images in advance and stored ina database such as a local descriptor DB shown in FIG. 1 or may beextracted on the fly from the reference image using the local descriptorextracting unit 11. When storing local descriptors in a database, localdescriptors extracted from reference images including similar objects(showing similar objects as subjects) may be registered in associationwith each other. Details of the local descriptor matching unit 12 willbe described later.

The input image difference area determining unit 13 performs a geometrictransformation indicated by the geometric transformation informationoutputted from the local descriptor matching unit 12 on a referenceimage corresponding to the local feature identified image ID outputtedfrom the local descriptor matching unit 12 or on a difference area of areference image group associated with the local feature identified imageID and outputs input image difference area information.

In the present embodiment, when it is predicted that a slight differencemay occur between an object shown in the input image and an object shownin the reference image, a difference area of the reference image refersto an area showing a portion in the reference image where the differencemay occur. For example, when a difference area of the reference image isrectangular, information on the difference area may be coordinate valueinformation of four corners of the rectangle. Alternatively, informationon the difference area of the reference image may be informationrepresenting coordinate values of a pixel group in a reference imagethat constitutes the difference area.

As the input image difference area information, a coordinate value in aninput image obtained by respectively performing a geometrictransformation on coordinate values of four corners of a difference areain the reference image may be adopted. Alternatively, when informationon a difference area in the reference image is coordinate valueinformation of a pixel group constituting the difference area, ageometric transformation corresponding to the geometric transformationinformation can be performed on each pixel in the pixel group andcoordinate value information of a pixel group constituting a differencearea in the input image can be adopted as input image difference areainformation.

Difference area information of the reference image is stored in adatabase in advance. For example, when the local descriptor 2 is storedin a database such as the local descriptor DB shown in FIG. 1, thedifference area information of the reference image may be stored in thelocal descriptor DB together with the local descriptor 2.

The input image difference area descriptor extracting unit 14 extracts adescriptor from an area in an input image (difference area in the inputimage) indicated by the input image difference area informationoutputted from the input image difference area determining unit 13.Details of the input image difference area descriptor extracting unit 14will be described later.

The descriptor matching unit 15 matches a descriptor 1 extracted fromthe difference area in the input image by the input image differencearea descriptor extracting unit 14 with a descriptor 2 extracted fromthe difference area in the reference image and outputs a matchingresult. In the matching, the descriptor matching unit 15 determineswhether an object included in the input image and an object included inthe reference image are the same (whether the input image and thereference image show a same object as a subject). When the objects aredetermined to be the same, the descriptor matching unit 15 outputs animage ID of the input image determined to be the same as a differencearea identified image ID.

The local descriptor 2 may be extracted from a plurality of referenceimages in advance and stored in a database as shown in FIG. 1 or may beextracted on the fly from a reference image. When storing the localdescriptor 2 in a database, a similar object may be associated andregistered with the local descriptor 2. Details of the descriptormatching unit 15 will be described later.

FIG. 2 is a flow chart showing a flow of processes by the input imagedifference area determining unit 13 shown in FIG. 1. As shown in FIG. 2,first, a variable i for controlling the processes is initialized inS131.

In S132, geometric transformation information outputted from the localdescriptor matching unit 12 is acquired. In S133, difference areainformation of a reference image is acquired from the local descriptorDB. When a difference area of the reference image is rectangular, thedifference area information acquired at this point may be coordinatevalue information of four corners of the rectangle or may be informationrepresenting coordinate values of a pixel group in a reference imageconstituting the difference area.

In S134, a geometric transformation indicated by the geometrictransformation information acquired in S132 is performed on thedifference area information acquired in S133. In this case, if thedifference area information is coordinate value information of the fourcorners, the geometric transformation is performed on one of the fourcoordinate values. In addition, if the difference area information iscoordinate value information of a pixel group in a reference imageconstituting the difference area, the geometric transformation isperformed on one pixel among the pixel group. At this point, when thevariable i is smaller than a prescribed number N, the value of thevariable i is updated in S135 and processes of S133 and S134 arecontinued until the value of the variable i equals or exceeds N. N=4 isset when the difference area information acquired in S133 is coordinatevalue information of the four corners in the reference image, and thenumber of pixels of a pixel group in a reference image constituting adifference area is set as the value of N when the difference areainformation is coordinate value information of the pixel group in thereference image constituting the difference area. Finally, in S136, theinput image difference area information calculated in S134 is outputtedand the process is finished.

Next, the local descriptor extracting unit 11 will be described indetail with reference to FIG. 3. FIG. 3 is a block diagram representinga configuration example of the local descriptor extracting unit 11. Thelocal descriptor extracting unit 11 includes a brightness informationextracting unit 101, a local feature point detecting unit 102, and alocal descriptor generating unit 103.

The brightness information extracting unit 101 receives an input imageand extracts and exclusively outputs information regarding brightnessfrom each pixel in the input image. In this case, the accepted inputimage is an image photographed by an imaging device such as a digitalcamera, a digital video camera, and a mobile phone, an image importedthrough a scanner, and the like. In addition, the image may be acompressed image such as a JPEG (Joint Photographic Experts Group) imageor an uncompressed image such as a TIFF (Tagged Image File Format)image.

The local feature point detecting unit 102 detects a large number ofcharacteristic points (feature points) from an image and outputs featurepoint information that is information related to each feature point. Inthis case, for example, feature point information refers to a coordinateposition or a scale of a detected feature point, an orientation of afeature point, a “feature point number” that is a unique ID(Identification) assigned to a feature point, or the like. Moreover, thelocal feature point detecting unit 102 may output the feature pointinformation as a separate piece of feature point information for eachdirection of an orientation of each feature point. For example, thelocal feature point detecting unit 102 may be configured to outputfeature point information only with respect to a direction of a mainorientation of each feature point or to also output feature pointinformation with respect to directions of secondary and subsequentorientations. In addition, when the local feature point detecting unit102 is configured to also output feature point information with respectto directions of secondary and subsequent orientations, the localfeature point detecting unit 102 can attach a different feature pointnumber to each direction of an orientation of each feature point. Forexample, the local feature point detecting unit 102 can use a DoG(Difference-of-Gaussian) process when detecting a feature point from animage and extracting feature point information. Specifically, the localfeature point detecting unit 102 can determine a position and a scale ofa feature point by using a DoG process to perform an extreme valuesearch in a scale space. Furthermore, the local feature point detectingunit 102 can calculate an orientation of each feature point using adetermined position and scale of a feature point and gradientinformation of a peripheral area. Moreover, the local feature pointdetecting unit 102 may use other methods such as the Fast-HessianDetector method instead of DoG to detect a feature point from an imageand extract feature point information. The local feature point detectingunit 102 may exclusively select important feature points from theinternally detected feature points and exclusively output informationrelated to the selected feature points as feature point information.

The local descriptor generating unit 103 receives the feature pointinformation outputted from the local feature point detecting unit 102and generates (describes) a local descriptor that is a descriptor of alocal area with respect to each feature point (an area including afeature point and a periphery thereof). Moreover, the local descriptorgenerating unit 103 may output a local descriptor in a losslesscompression format such as ZIP and LZH. When an importance of detectedfeature points is determined by the local feature point detecting unit102, the local descriptor generating unit 103 can generate and output alocal descriptor in an order of importance of the feature points.Alternatively, the local descriptor generating unit 103 may generate andoutput a local descriptor in an order of coordinate positions of featurepoints. First, based on descriptor information, the local descriptorgenerating unit 103 acquires a local area from which a descriptor is tobe extracted based on a coordinate position, a scale, and an orientationof each detected feature point. Moreover, when a plurality of pieces offeature point information with different orientations exist for onefeature point, a local area can be acquired with respect to each pieceof feature point information. Next, after normalizing a local area byrotating the local area in accordance with an orientation direction of afeature point, the local area is divided into sub-areas. For example, alocal area can be divided into 16 blocks (4×4 blocks). Next, a featurevector is generated for each sub-area of the local area. For example, agradient direction histogram can be used as a feature vector of asub-area. Specifically, a gradient direction histogram is generated bycalculating a gradient direction for each pixel in each sub-area,quantizing the gradient direction into eight directions, and summing upfrequencies of the eight quantized directions for each sub-area. At thispoint, a feature vector constituted by a gradient direction histogram of16 blocks×8 directions which is generated with respect to each featurepoint is outputted as a local descriptor. The outputted local descriptoris outputted so as to include coordinate position information of afeature point.

Next, the local descriptor matching unit 12 will be described in detailwith reference to FIG. 4. FIG. 4 is a block diagram representing aconfiguration example of the local descriptor matching unit 12. As shownin FIG. 4, the local descriptor matching unit 12 includes acorresponding feature point determining unit 201, an erroneouscorresponding point removing unit 202, an identification scorecalculating unit 203, and a threshold determining unit 204.

The corresponding feature point determining unit 201 receives the localdescriptor 1 extracted from an input image by the local descriptorextracting unit 11 and the local descriptor 2 extracted from a referenceimage. The corresponding feature point determining unit 201 determineswhether or not the local descriptor 1 and the local descriptor 2correspond to each other and, if so, outputs corresponding feature pointinformation describing that the local descriptor 1 and the localdescriptor 2 correspond to each other. For example, when the localdescriptor 1 and the local descriptor 2 are, respectively, sets of adescriptor describing a gradient histogram of a periphery of a featurepoint, distance calculations in a descriptor space are first performedfor all combinations of local descriptors. Only when a minimum distancevalue is significantly smaller than a next small distance value, withrespect to a combination of a local descriptor that produces the minimumdistance value, a determination is made that the local descriptor and alocal feature area of the local descriptor correspond to each other andposition information of the local feature area and position informationof the corresponding local feature area are outputted as correspondingfeature point information.

The erroneous corresponding point removing unit 202 receives thecorresponding feature point information from the corresponding featurepoint determining unit 201, distinguishes correctly correspondingfeature points from erroneously-corresponding feature points among thecorresponding feature points, and respectively outputs distinguishedfeature point information together with geometric transformationinformation used for the distinguishment. For example, a method such asRANSAC is applied to corresponding feature point information receivedfrom the corresponding feature point determining unit 201 andinformation on a geometric transformation that moves a coordinate in thereference image to a coordinate in the input image is estimated asgeometric transformation information. The geometric transformationinformation estimated at this point is respectively applied to featurepoints in the reference image among the corresponding feature points, adetermination of a correctly corresponding feature point is made when afeature point in the reference image is substantially consistent with afeature point in the input image, and a determination of anerroneously-corresponding feature point is conversely made when afeature point in the reference image is not consistent with a featurepoint in the input image.

The identification score calculating unit 203 receives the correspondingfeature point information from the erroneous corresponding pointremoving unit 202 and outputs an identification score. As the outputtedidentification score, for example, the number of combinations ofcorrectly corresponding feature points may be counted from among thecorresponding feature point information received from the erroneouscorresponding point removing unit 202, a table for mapping the number ofcombinations to a score ranging from 0 to 1 may be prepared in advance,and the identification score may be outputted by referring to the table.Alternatively, when the number of combinations of correctlycorresponding feature points is denoted by c, by denoting a minimumnumber of corresponding feature points set in advance as m, m/(c+m) maybe calculated as the identification score. The threshold determiningunit 204 performs a threshold process on the identification scoreoutputted from the identification score calculating unit 203, makes adetermination that the images show a same object when the identificationscore is equal to or higher than the threshold, and outputs the ID ofthe reference image as a local feature identified image ID. Thethreshold set by the threshold determining unit 204 may be a valuedetermined and internally stored in advance or a value provided from theoutside.

Next, the input image difference area descriptor extracting unit 14 willbe described in detail with reference to FIG. 5. FIG. 5 is a blockdiagram representing a configuration example of the input imagedifference area descriptor extracting unit 14. As shown in FIG. 5, theinput image difference area descriptor extracting unit 14 includes adifference area image generating unit 401 and a difference areadescriptor calculating unit 402.

The difference area image generating unit 401 receives an input imageand receives input image difference area information from the inputimage difference area determining unit 13, and when the input imagedifference area information is coordinate value information of fourcorners of a difference area in the input image, respectively connectstwo adjacent corners among the four corners by straight lines andsequentially reads pixels on the straight lines. A difference area imagein the input image is generated and outputted by determining pixels forwhich a value is to be read and an order in which the pixels are to beread from the input image with respect to an area enclosed by a readpixel group. Alternatively, when the input image difference areainformation received from the input image difference area determiningunit 13 is information representing coordinate values of a pixel groupconstituting a difference area in the input image, the difference areaimage generating unit 401 reads the input image in the order ofcoordinate values and outputs the read results as a difference areainformation in the input image.

The difference area descriptor calculating unit 402 extracts adescriptor from the difference area image generated by the differencearea image generating unit 401 and outputs the descriptor. As thedescriptor extracted by the difference area descriptor calculating unit402, for example, a descriptor such as “color arrangement” and a “colorhistogram” may be extracted in order to perform an analysis on colorinformation of a difference area in the input image and a differencearea in the reference image. Alternatively, a descriptor capable ofexpressing “character-likeness” may be extracted in order to analyzeminute differences in characters between a difference area in the inputimage and a difference area in the reference image.

Next, the descriptor matching unit 15 will be described in detail withreference to FIG. 6. FIG. 6 is a block diagram representing aconfiguration example of the descriptor matching unit 15. As shown inFIG. 6, the descriptor matching unit 15 includes a difference areaidentification score calculating unit 501 and a threshold determiningunit 502.

The difference area identification score calculating unit 501respectively receives a descriptor 1 as a descriptor extracted from adifference area of the input image and a descriptor 2 as a descriptorextracted from a difference area of the reference image. The differencearea identification score calculating unit 501 outputs an identificationscore determined from the two received descriptors as a difference areaidentification score. The difference area identification score is ascale whose value increases as the similarity between the descriptor 1and the descriptor 2 increases. For example, a distance between thedescriptor 1 and the descriptor 2 on a descriptor space may becalculated and an inverse thereof may be outputted as a difference areaidentification score. Alternatively, when matching the descriptor 1 withdescriptors 2 respectively extracted from a plurality of reference imagegroups, a minimum value of distances on the descriptor space among alldescriptor combinations may be found and an inverse of a value obtainedby dividing distances on the descriptor space for all descriptorcombinations by the minimum value may be outputted as a difference areaidentification score. Alternatively, a table for mapping distance valuesbetween the descriptor 1 and the descriptor 2 on the descriptor space toscores ranging from 0 to 1 may be prepared in advance and a differencearea identification score may be outputted by referring to the table.

The threshold determining unit 502 compares the difference areaidentification score outputted from the difference area identificationscore calculating unit 501 with a threshold, makes a determination thatimages show a same object when the identification score is equal to orhigher than the threshold, and outputs the ID of the reference image asa difference area identified image ID. The threshold set by thethreshold determining unit 502 may be a value determined and internallystored in advance or a value provided from the outside.

FIG. 7 shows a configuration of an object identification apparatusaccording to a modification of the present embodiment. The objectidentification apparatus shown in FIG. 7 differs from the objectidentification apparatus shown in FIG. 1 in that a difference areainformation DB that is a database exclusively storing difference areainformation is provided. The present embodiment can be realized by theconfiguration shown in FIG. 7 when the local descriptor 2 is not storedin the form of a database and can be extracted on the fly from areference image.

FIGS. 8 to 11 are conceptual diagrams showing patterns that areconceivable as relationships between a difference area in a referenceimage and a difference area in an input image.

FIG. 8 shows an example of a case in which an object is displayed acrossan entire reference image and a difference area is set with respect tothe entire reference image. This example corresponds to cases in which,for example, characters, patterns, or the like inscribed on objects aresubstantially the same but colors of the objects differ from each otheras often seen in a case of, for example, packages of candy that belongto a same brand but have different flavors. In other words, in thisexample, since it is conceivable that an entire input image may differfrom an entire reference image, the entire reference image is set as adifference area.

FIG. 9 shows an example of a case in which an object is displayed acrossan entire reference image and a difference area is set with respect to apart of the reference image. This example corresponds to cases in which,for example, objects are substantially the same but colors, characters,patterns, or the like differ from each other only in a certain part ofthe objects as often seen in a case of, for example, spines of booksrepresenting different volumes of a same series. In other words, in thisexample, since it is conceivable that a part of an input image maydiffer from a part of a reference image, a part of the reference imageis set as a difference area. The examples shown in FIGS. 10 and 11 aresubstantially similar to the examples shown in FIGS. 8 and 9 but differin that, instead of showing a reference image in its entirety, only apart of a reference image is shown.

In every case, in the present embodiment, information on an area set asthe difference area is registered in a database in advance. In thepresent embodiment, a registered difference area in a reference imageand a registered difference area in an input image are respectivelyobtained by extracting minimum required areas in which it is predictedthat a difference may occur from the reference image and the inputimage. Therefore, even in a case where a local descriptor of the entirereference image and a local descriptor of the entire input image arecompared to each other and the images are determined to be the samesince there is only a slight difference, by once again exclusivelycomparing descriptors of difference images of the reference image andthe input image, a minute difference attributable to a differencebetween objects can be distinguished. As a result, erroneousdeterminations that have been a problem when only using localdescriptors can be suppressed.

As described above, according to the present embodiment, the localdescriptor matching unit 12 determines whether or not a descriptor ofeach feature point extracted from an input image corresponds to adescriptor of each feature point extracted from a reference image. Whena score based on the number of combinations of descriptors determined tobe corresponding descriptors by the local descriptor matching unit 12 isequal to or higher than a prescribed value, the input image differencearea descriptor extracting unit 14 extracts a descriptor from an area ofthe input image corresponding to a position of an image area obtained byperforming a geometric transformation for correcting a geometricdeviation between the input image and the reference image on aprescribed area (difference area) of the reference image. Moreover, inthe present embodiment, a difference area of the reference image isdetermined by the input image difference area determining unit 13. Thedescriptor matching unit 15 matches a descriptor extracted by the inputimage difference area descriptor extracting unit 14 with a descriptorextracted from the difference area of the reference image and outputs amatching result. As a result, an input image showing a same object as anobject shown in a reference image can be identified more accurately.

Second Embodiment

A second embodiment of the present invention will be described withreference to the drawings. In the second embodiment, a difference areain a reference image is estimated and identification is performedwithout registering the difference area in the reference image in adatabase in advance.

FIG. 12 is a block diagram showing a configuration of an objectidentification apparatus according to the second embodiment of thepresent invention. As shown in FIG. 12, the object identificationapparatus according to the present embodiment includes a localdescriptor extracting unit 11, a local descriptor matching unit 16, aninput image difference area determining unit 13, an input imagedifference area descriptor extracting unit 14, a difference areaestimating unit 17, a difference area descriptor extracting unit 18, anda descriptor matching unit 15. As shown, the object identificationapparatus according to the second embodiment differs from the objectidentification apparatus according to the first embodiment in that thelocal descriptor matching unit 12 has been changed to the localdescriptor matching unit 16 and the difference area information DB thatis a database storing difference area information has been changed tothe difference area estimating unit 17 and the difference areadescriptor extracting unit 18. Details of the local descriptor matchingunit 16, the difference area estimating unit 17, and the difference areadescriptor extracting unit 18 will be described later. Since othercomponents are similar to those of the first embodiment, the componentswill be denoted by same reference symbols and a detailed descriptionthereof will be omitted.

FIG. 13 is a block diagram representing a configuration example of thelocal descriptor matching unit 16. As shown in FIG. 13, the localdescriptor matching unit 16 includes a corresponding feature pointdetermining unit 201, an erroneous corresponding point removing unit202, an identification score calculating unit 203, and a thresholddetermining unit 204. In other words, the components of the localdescriptor matching unit 16 shown in FIG. 13 are the same as thecomponents of the local descriptor matching unit 12 shown in FIG. 4.However, the local descriptor matching unit 16 shown in FIG. 13 differsfrom the local descriptor matching unit 12 shown in FIG. 4 in that, inaddition to geometric transformation information being outputted fromthe erroneous corresponding point removing unit 202 and a local featureidentified image ID being outputted from the threshold determining unit204, corresponding feature point information outputted from theerroneous corresponding point removing unit 202 is now outputted fromthe local descriptor matching unit 16.

FIG. 14 is a block diagram representing a configuration example of thedifference area estimating unit 17. As shown in FIG. 14, the differencearea estimating unit 17 includes an erroneously-corresponding featurepoint concentration searching unit 701. The erroneously-correspondingfeature point concentration searching unit 701 receives correspondingfeature point information from the local descriptor matching unit 16 andoutputs difference area information that is information regarding adifference area in a reference image. The corresponding feature pointinformation received from the local descriptor matching unit 16 includesinformation on correctly corresponding feature points and information onerroneously-corresponding feature points. Therefore, by searching anarea with a concentration of erroneously-corresponding feature pointsfrom the reference image using information on theerroneously-corresponding feature points included in the correspondingfeature point information, a difference area in the reference image canbe estimated. To search an area with a concentration oferroneously-corresponding feature points, for example, when arectangular window with a certain size is defined, the rectangularwindow is moved in the difference image, and the number oferroneously-corresponding feature points in the rectangular window isequal to or larger than a certain number, an area of the rectangularwindow can be assumed to be a difference area. A method of estimating adifference area in a reference image is not limited thereto and anyestimation method can be used as long as the estimation is based on anarea with a concentration of erroneously-corresponding feature points.In other words, a difference area (prescribed area) of a reference imageis an area including an area of the reference image that is determinedby the local descriptor matching unit 16 to have a concentration oferroneously-corresponding feature points.

FIG. 15 is a block diagram representing a configuration example of thedifference area descriptor extracting unit 18. As shown in FIG. 15, thedifference area descriptor extracting unit 18 includes a difference areaimage generating unit 801 and a difference area descriptor calculatingunit 402.

The difference area image generating unit 801 is substantially the sameas the difference area image generating unit 401 that is a component ofthe input image difference area descriptor extracting unit 14 shown inFIG. 5. However, the difference area image generating unit 801 differsfrom the difference area image generating unit 401 in that a referenceimage and difference area information are respectively inputted insteadof an input image and input image difference area information. Anotherdifference is that a difference image generated by the difference areaimage generating unit 801 is generated from a reference image based ondifference area information in the reference image instead of beinggenerated from an input image based on input image difference areainformation.

Since the difference area descriptor calculating unit 402 is the same asthe difference area descriptor calculating unit 402 that is a componentof the input image difference area descriptor extracting unit 14 shownin FIG. 5, a detailed description thereof will be omitted. Moreover,descriptors calculated by the difference area descriptor calculatingunits 402 in FIGS. 5 and 15 must be descriptors calculated by a sameprocess.

As described above, according to the present embodiment, since adifference area in a reference image can be estimated even withouthaving to register the difference area in the reference image in adatabase in advance, the present embodiment is effective when an area inwhich it is predicted that a difference may occur cannot be registeredin advance as a difference area in an article inspection system thatutilizes object identification (for example, when exclusivelydistinguishing products with a defect in one part or another from amonga large number of products). In addition, since estimation of adifference area in the reference image according to the presentembodiment can be performed regardless of whether the difference areacovers an entire object or the difference area corresponds to a part ofthe object, the present embodiment is effective with respect to any ofthe examples shown in FIGS. 8 to 11.

Third Embodiment

A third embodiment of the present invention will be described withreference to the drawings.

FIG. 16 is a block diagram showing a configuration of an objectidentification apparatus according to the third embodiment of thepresent invention. As shown in FIG. 16, the object identificationapparatus according to the third embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 16, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 19, and a difference area descriptorextracting unit 18.

As described above, the object identification apparatus according to thethird embodiment differs from the second embodiment in that thedifference area estimating unit 17 of the object identificationapparatus according to the second embodiment has been changed to thedifference area estimating unit 19. Details of the difference areaestimating unit 19 will be described later. Since other components aresimilar to those of the second embodiment, the components will bedenoted by same reference symbols and a detailed description thereofwill be omitted.

FIG. 17 is a block diagram representing a configuration example of thedifference area estimating unit 19. As shown in FIG. 17, the differencearea estimating unit 19 includes an object area estimating unit 901 andan erroneously-corresponding feature point concentration searching unit902.

The object area estimating unit 901 receives a reference imagecorresponding to a local feature identified image ID outputted from thelocal descriptor matching unit 16 or a reference image group associatedwith the local feature identified image ID and outputs object areainformation that is information representing an area in which an objectexists in a reference image. The reference image received at this pointmay be stored in advance in a database as shown in FIG. 16 or may beacquired from outside of the object identification apparatus.Conceivable examples of processes by the object area estimating unit 901include a method of roughly estimating an object area by analyzing edgeintensity in a reference image and a method involving learning an imagepattern of a background area in advance and roughly estimating an objectarea as an area other than a background.

The erroneously-corresponding feature point concentration searching unit902 is similar to the erroneously-corresponding feature pointconcentration searching unit 701 that is a component of the differencearea estimating unit 17 shown in FIG. 14. However, theerroneously-corresponding feature point concentration searching unit 902differs from the second embodiment in that object area informationoutputted from the object area estimating unit 901 is inputted inaddition to corresponding feature point information received from thelocal descriptor matching unit 16. The erroneously-corresponding featurepoint concentration searching unit 902 only focuses on points inside anobject area among corresponding feature points and searches for an areawith a concentration of erroneously-corresponding feature points.

In other words, the erroneously-corresponding feature pointconcentration searching unit 902 estimates a difference area from insidean object area in a reference image. Therefore, in the presentembodiment, a difference area in the reference image can be estimatedwithout being affected by an erroneously-corresponding feature pointthat appears from an area other than the object. Specifically, adifference area (prescribed area) in the reference image is an areaincluding an area that is determined by the local descriptor matchingunit 16 to have a concentration of erroneously-corresponding featurepoints among an area showing an object among the reference image. Inaddition, since a range in which an area with a concentration oferroneously-corresponding feature points is to be searched among thereference image is limited, the erroneously-corresponding feature pointconcentration searching unit 902 is capable of performing a process at ahigher speed than the erroneously-corresponding feature pointconcentration searching unit 701 whose search range is an entirereference image.

According to the present embodiment, in a similar manner to the secondembodiment, since a difference area in a reference image can beestimated even without having to register the difference area in thereference image in a database in advance, the present embodiment iseffective when information regarding a difference area cannot beregistered in advance in an article inspection system that utilizesobject identification (for example, when exclusively distinguishingproducts with a defect in one part or another from among a large numberof products). In addition, while estimation of a difference area in thereference image according to the present embodiment can be performedregardless of whether the difference area is an entire object or a partof the object, since the difference area can be estimated with highaccuracy without being affected by an erroneously-corresponding featurepoint that appears from an area other than the object, the presentembodiment is particularly effective in cases of the examples shown inFIGS. 10 and 11.

Fourth Embodiment

A fourth embodiment of the present invention will be described withreference to the drawings. In the fourth embodiment, a case in whichanother method is used as a difference area estimation method will bedescribed.

FIG. 18 is a block diagram showing a configuration of an objectidentification apparatus according to the fourth embodiment of thepresent invention. As shown in FIG. 18, the object identificationapparatus according to the fourth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 12, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 20, and a difference area descriptorextracting unit 18. As described above, the object identificationapparatus according to the fourth embodiment differs from the thirdembodiment in that the local descriptor matching unit 16 and thedifference area estimating unit 19 of the object identificationapparatus according to the third embodiment has been changed to thelocal descriptor matching unit 12 and the difference area estimatingunit 20. The local descriptor matching unit 12 is similar to the localdescriptor matching unit 12 of the object identification apparatusaccording to the first embodiment and a detailed description thereofwill be omitted. Details of the difference area estimating unit 20 willbe described later. Since other components are similar to those of thethird embodiment, the components will be denoted by same referencesymbols and a detailed description thereof will be omitted.

FIG. 19 is a block diagram representing a configuration example of thedifference area estimating unit 20. As shown in FIG. 19, the differencearea estimating unit 20 includes a converted image generating unit 2001,a difference image generating unit 2002, an object area estimating unit2003, and a large difference area detecting unit 2004.

The converted image generating unit 2001 receives an input image andgeometric transformation information outputted from the local descriptormatching unit 12, performs a geometric transformation indicated by thegeometric transformation information on the input image, and outputs aconverted image. The converted image outputted at this point isgenerated by, for example, performing a geometric transformationindicated by the geometric transformation information on each pixel inthe input image and projecting each pixel in the input image onto animage with a same size as a reference image. In this case, pixel valuesof pixels in the projection destination image onto which a pixel in theinput image is not projected are filled with zero or the like togenerate the converted image. In addition, when the geometrictransformation information outputted from the local descriptor matchingunit 12 is information with respect to a transformation of a coordinatein the reference image to a coordinate in the input image, the geometrictransformation information to be applied by the converted imagegenerating unit 2001 must be information for performing an oppositetransformation. Specifically, when the geometric transformationinformation outputted from the local descriptor matching unit 12 is a3×3 matrix that converts a coordinate in the reference image to acoordinate in the input image, an inverse matrix thereof is used as thegeometric transformation information to be applied by the convertedimage generating unit 2001.

The difference image generating unit 2002 receives a reference imagecorresponding to a local feature identified image ID outputted from thelocal descriptor matching unit 12 (or a reference image group associatedwith the local feature identified image ID) and the converted imageoutputted from the converted image generating unit 2001. The differenceimage generating unit 2002 outputs an image representing a differencebetween the reference image and the converted image as a differenceimage. Moreover, when calculating the difference between the referenceimage and the converted image, for example, the difference may becalculated after correcting brightness of one of the images so thataverage values of brightness of both images are consistent with eachother. The reference image received at this point may be stored inadvance in a database as shown in FIG. 18 or may be acquired fromoutside of an article identification apparatus.

The object area estimating unit 2003 receives the difference image fromthe difference image generating unit 2002 and estimates and outputsobject area information (information representing an area in which anobject exists in the reference image) in the difference image. Theobject area information outputted at this point can be estimated by, forexample, searching an area with a small difference value among thedifference value from the image. This is because an area with a smalldifference value among the difference value is conceivably an area inwhich it is highly likely that a same object is shown in the referenceimage and the converted image. To estimate an object area, for example,when a rectangular window with a certain size is considered, therectangular window is moved in the difference image, and the number ofpixels with small pixel values in the rectangular window is equal to orlarger than a certain number, an area of the rectangular window can beassumed to be an object area. Other methods may be used instead.

The large difference area detecting unit 2004 receives the differenceimage outputted from the difference image generating unit 2002 and theobject area information outputted from the object area estimating unit2003. The large difference area detecting unit 2004 determines that alocation with an increased difference value in the difference imageamong the object area is highly likely to be a location where there is adifference between the object shown in the reference image and theobject shown in the converted image. Therefore, the large differencearea detecting unit 2004 searches a location with a large differencevalue from among the image and outputs area information of the locationas difference area information. To search a location with a largedifference value (in other words, a difference area), for example, whena rectangular window with a certain size is defined, the rectangularwindow is moved in an object area of the difference image, and thenumber of pixels with large pixel values in the rectangular window isequal to or larger than a certain number, an area of the rectangularwindow can be assumed to be a difference area. Other methods may be usedinstead.

As described above, according to the present embodiment, the convertedimage generating unit 2001 performs a geometric transformation forcorrecting a geometric deviation between an input image and a referenceimage on the input image and outputs a converted image. The largedifference area detecting unit 2004 outputs information on an areaincluding an area in which a difference between the converted image andthe reference image is equal to or larger than a prescribed value asdifference area information. In addition, the large difference areadetecting unit 2004 can output information on an area including an areain which a difference between the converted image and the referenceimage is equal to or larger than a prescribed value among an area inwhich an object exists among the reference image as difference areainformation.

According to the present embodiment, in a similar manner to the secondand third embodiments, since a difference area in a reference image canbe estimated even without having to register the difference area in thereference image in a database in advance, the present embodiment iseffective when information regarding a difference area cannot beregistered in advance in an article inspection system that utilizesobject identification (for example, when exclusively distinguishingproducts with a defect in one part or another from among a large numberof products). In addition, while estimation of a difference area in thereference image according to the present embodiment can be performedregardless of whether the difference area is an entire object or a partof the object, since the difference area is estimated after removing theinfluence of a background by first estimating an object area in asimilar manner to the third embodiment, the difference area can beestimated with high accuracy. Accordingly, the present embodiment isparticularly effective in cases such as the examples shown in FIGS. 10and 11.

Fifth Embodiment

A fifth embodiment of the present invention will be described withreference to the drawings.

FIG. 20 is a block diagram showing a configuration of an objectidentification apparatus according to the fifth embodiment of thepresent invention. As shown in FIG. 20, the object identificationapparatus according to the fifth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 16, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 21, and a difference area descriptorextracting unit 18. As described above, the object identificationapparatus according to the fifth embodiment is configured so as tocombine the object identification apparatus according to the thirdembodiment and the object identification apparatus according to thefourth embodiment. The object identification apparatus according to thefifth embodiment differs from the object identification apparatusaccording to the third embodiment in that the difference area estimatingunit 19 has been changed to the difference area estimating unit 21.Details of the difference area estimating unit 21 will be describedlater. Since other components are similar to those of the thirdembodiment, the components will be denoted by same reference symbols anda detailed description thereof will be omitted.

FIGS. 21 to 24 are block diagrams representing configuration examples ofthe difference area estimating unit 21. Each drawing will be describedbelow.

The difference area estimating unit 21 shown in FIG. 21 includes aconverted image generating unit 2001, a difference image generating unit2002, an object area estimating unit 2003, and anerroneously-corresponding feature point concentration searching unit902. The converted image generating unit 2001, the difference imagegenerating unit 2002, and the object area estimating unit 2003 shown inFIG. 21 are the same as the converted image generating unit 2001, thedifference image generating unit 2002, and the object area estimatingunit 2003 that are components of the difference area estimating unit 20shown in FIG. 19 and a detailed description thereof will be omitted. Inaddition, the erroneously-corresponding feature point concentrationsearching unit 902 shown in FIG. 21 is the same as theerroneously-corresponding feature point concentration searching unit 902that is a component of the difference area estimating unit 19 shown inFIG. 17 and a detailed description thereof will be omitted.

Specifically, instead of searching for an area with a concentration oferroneously-corresponding feature points among an object area that isestimated by only using a reference image as in the case of thedifference area estimating unit 19, the difference area estimating unit21 estimates a difference area by searching for an area with aconcentration of erroneously-corresponding feature points among anobject area that is estimated by using a difference between a convertedinput image and a reference image.

In addition, the difference area estimating unit 21 shown in FIG. 22includes a converted image generating unit 2001, a difference imagegenerating unit 2002, an object area estimating unit 2003, a largedifference area detecting unit 2101, and an erroneously-correspondingfeature point concentration searching unit 2102. The converted imagegenerating unit 2001, the difference image generating unit 2002, and theobject area estimating unit 2003 shown in FIG. 22 are the same as theconverted image generating unit 2001, the difference image generatingunit 2002, and the object area estimating unit 2003 that are componentsof the difference area estimating unit 20 shown in FIG. 19 and adetailed description thereof will be omitted.

The large difference area detecting unit 2101 shown in FIG. 22 issubstantially the same as the large difference area detecting unit 2004that is a component of the difference area estimating unit 20 shown inFIG. 19 but differs in that the large difference area detecting unit2101 outputs difference candidate area information instead of differencearea information. The difference candidate area information outputted bythe large difference area detecting unit 2101 may be the same asdifference area information outputted by the large difference areadetecting unit 2004 or area information representing an area that isslightly larger than the difference area information may be adoptedinstead.

The erroneously-corresponding feature point concentration searching unit2102 shown in FIG. 22 is similar to the erroneously-correspondingfeature point concentration searching unit 902 that is a component ofthe difference area estimating unit 19 shown in FIG. 17 but differs inthat difference candidate area information is inputted instead of objectarea information. For the difference area information that is outputtedfrom the erroneously-corresponding feature point concentration searchingunit 2102, since a difference area is further narrowed down by theerroneously-corresponding feature point concentration searching unit2102 from difference candidate areas estimated by a combination of fourunits, namely, the converted image generating unit 2001, the differenceimage generating unit 2002, the object area estimating unit 2003, andthe large difference area detecting unit 2101. Therefore, highlyaccurate difference area information is outputted.

In addition, the difference area estimating unit 21 shown in FIG. 23includes a converted image generating unit 2001, anerroneously-corresponding feature point concentration searching unit2103, a difference image generating unit 2104, and a large differencearea detecting unit 2105. The converted image generating unit 2001 shownin FIG. 23 is the same as the converted image generating unit 2001 thatis a component of the difference area estimating unit 20 shown in FIG.19 and a detailed description thereof will be omitted.

The erroneously-corresponding feature point concentration searching unit2103 shown in FIG. 23 is substantially the same as theerroneously-corresponding feature point concentration searching unit 701that is a component of the difference area estimating unit 17 shown inFIG. 14 but differs in that the erroneously-corresponding feature pointconcentration searching unit 2103 outputs difference candidate areainformation instead of difference area information. The differencecandidate area information outputted by the erroneously-correspondingfeature point concentration searching unit 2103 may be the same asdifference area information outputted by the erroneously-correspondingfeature point concentration searching unit 701 or area informationrepresenting an area slightly larger than the difference areainformation may be adopted instead.

The difference image generating unit 2104 shown in FIG. 23 is similar tothe difference image generating unit 2002 that is a component of thedifference area estimating unit 20 shown in FIG. 19 but differs in thatdifference candidate area information is inputted in addition to areference image and a converted image. The difference image generatingunit 2104 outputs, as a difference image, an image of an area indicatedby difference candidate area information from a difference imagegenerated by calculating a difference between the reference image andthe converted image.

The large difference area detecting unit 2105 shown in FIG. 23 issimilar to the large difference area detecting unit 2004 that is acomponent of the difference area estimating unit 20 shown in FIG. 19 butdiffers in that only a difference image is inputted. Since thedifference image inputted to the large difference area detecting unit2105 is a difference image outputted only with respect to an area thatis already estimated to be a difference candidate area by theerroneously-corresponding feature point concentration searching unit2103, even at this stage, the image already shows an entire object areaor a part of an object area. Since the difference area informationoutputted from the large difference area detecting unit 2105 representsa difference area further narrowed down by the large difference areadetecting unit 2402 from difference candidate areas estimated by theerroneously-corresponding feature point concentration searching unit2103, difference area information with high reliability is outputted.

In addition, the difference area estimating unit 21 shown in FIG. 24includes a converted image generating unit 2001, a difference imagegenerating unit 2002, an object area estimating unit 2003, a largedifference area detecting unit 2101, an erroneously-correspondingfeature point concentration searching unit 2103, and a differencecandidate area overlap detecting unit 2106. The converted imagegenerating unit 2001, the difference image generating unit 2002, and theobject area estimating unit 2003 shown in FIG. 24 are the same as theconverted image generating unit 2001, the difference image generatingunit 2002, and the object area estimating unit 2003 that are componentsof the difference area estimating unit 20 shown in FIG. 19 and adetailed description thereof will be omitted. The large difference areadetecting unit 2101 shown in FIG. 24 is the same as the large differencearea detecting unit 2101 shown in FIG. 22 and a detailed descriptionthereof will be omitted. In addition, the erroneously-correspondingfeature point concentration searching unit 2103 shown in FIG. 24 is thesame as the erroneously-corresponding feature point concentrationsearching unit 2103 shown in FIG. 23 and a detailed description thereofwill be omitted.

The difference candidate area overlap detecting unit 2106 shown in FIG.24 receives the difference candidate area information outputted from thelarge difference area detecting unit 2101 and the difference candidatearea information outputted from the erroneously-corresponding featurepoint concentration searching unit 2103, determines that an overlappingarea of the two difference candidate areas to be a difference area, andoutputs difference area information thereof. Since the difference areainformation outputted from the difference candidate area overlapdetecting unit 2106 is information regarding an area determined to be adifference candidate area by both the large difference area detectingunit 2101 and the erroneously-corresponding feature point concentrationsearching unit 2103, highly reliable difference area information isoutputted.

According to the present embodiment, in a similar manner to the second,third, and fourth embodiments, since a difference area in a referenceimage can be estimated even without having to register the differencearea in the reference image in a database in advance, the presentembodiment is effective in cases where information regarding adifference area cannot be registered in advance in an article inspectionsystem that utilizes object identification such as when exclusivelydistinguishing products with a defect in one part or another from amonga large number of products. In addition, while a difference area in areference image can be estimated according to the present embodimentregardless of whether the difference area is an entire object or a partof the object, since a highly reliable difference area can be obtainedas compared to the second embodiment and the like, highly accurateidentification can be achieved. Moreover, in the present embodiment, anobject area estimating unit may be added in front of theerroneously-corresponding feature point concentration searching unit2103 if the difference area estimating unit 21 is configured as shown inFIG. 23 or 24. In such a case, a difference candidate area is to beestimated by performing an erroneously-corresponding feature pointconcentration search among an estimated object area. At this point,since a difference area can be estimated after first removing theinfluence of a background in a similar manner to the third and fourthembodiments, the present embodiment is particularly effective in casessuch as the examples shown in FIGS. 10 and 11.

Sixth Embodiment

A sixth embodiment of the present invention will be described withreference to the drawings.

FIG. 25 is a block diagram showing a configuration of an objectidentification apparatus according to the sixth embodiment of thepresent invention. As shown in FIG. 25, the object identificationapparatus according to the sixth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 12, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 22, and a difference area descriptorextracting unit 18. As described above, the object identificationapparatus according to the sixth embodiment differs from the fourthembodiment in that the difference area estimating unit 20 of the objectidentification apparatus according to the fourth embodiment has beenchanged to the difference area estimating unit 22. Details of thedifference area estimating unit 22 will be described later. Since othercomponents are similar to those of the fourth embodiment, the componentswill be denoted by same reference symbols and a detailed descriptionthereof will be omitted.

FIGS. 26 and 27 are block diagrams representing configuration examplesof the difference area estimating unit 22. Each drawing will bedescribed below.

As shown in FIG. 26, the difference area estimating unit 22 includes atemplate matching unit 2201. The template matching unit 2201 receives areference image corresponding to a local feature identified image IDoutputted from the local descriptor matching unit 12 (or a referenceimage group associated with the local feature identified image ID) and,at the same time, receives a template image of a difference areacorresponding to the local feature identified image ID outputted fromthe local descriptor matching unit 12 (or a template image groupassociated with the local feature identified image ID), and outputsdifference area information based on the received image group.Specifically, the template image is an image pattern that is typicallyobserved in a periphery of a difference area. The template matching unit2201 estimates a difference area in a reference image by performing atemplate matching process in which each area in the reference image ismatched with the template image and an area most similar to the templateimage is searched. In other words, an area whose degree of similaritywith a prescribed pattern image is equal to or higher than a prescribedvalue among the reference image is set as the difference area in thereference image. The template image may be stored in advance in adatabase as shown in FIG. 25 or may be acquired from outside of theobject identification apparatus.

Alternatively, as a modification, the difference area estimating unit 22may include an object area estimating unit 901 and a template matchingunit 2202 as shown in FIG. 27. The object area estimating unit 901 shownin FIG. 27 is the same as the object area estimating unit 901 that is acomponent of the difference area estimating unit 19 shown in FIG. 17 anda detailed description thereof will be omitted.

The template matching unit 2202 is similar to the template matching unit2201 that is a component of the difference area estimating unit 22 shownin FIG. 26. The template matching unit 2202 differs from the templatematching unit 2201 in that object area information outputted from theobject area estimating unit 901 is inputted in addition to the referenceimage corresponding to the local feature identified image ID outputtedfrom the local descriptor matching unit 12 and the template image of adifference area corresponding to the local feature identified image IDoutputted from the local descriptor matching unit 12. The templatematching unit 2202 can estimate a difference area in a reference imageby performing template matching using a template image only on an objectarea in the reference image. In addition, with the template matchingunit 2202, since a range of an area in the reference image to be matchedwith the template image is limited, the template matching unit 2202 iscapable of performing processes at a higher speed than the templatematching unit 2201 which matches a range of an area equal to the entirereference image with the template image.

In the present embodiment, while a difference area in a reference imageneed not be registered in advance in a database in a similar manner tothe second to fifth embodiments, if an image pattern typically observedin the difference area is known in advance, the difference area can beestimated by using the image pattern as a template image. For example,when exclusively identifying specific mail from among a plurality ofmail images which show a same envelope and only differ from one anotherin addresses, an area where the address is described can be defined asan image pattern in which a layout of character strings representing apostal code, an address, an addressee, and the like is more or lessdetermined. Therefore, in such cases, the present embodiment iseffective. In addition, while estimation of a difference area in thereference image according to the present embodiment can be performedregardless of whether the difference area is an entire object or a partof the object, since the difference area can be estimated after removingthe influence of a background by first estimating an object area in asimilar manner to the third to fifth embodiments when the differencearea estimating unit 22 is configured as shown in FIG. 27, thedifference area can be estimated with high accuracy. Accordingly, thepresent embodiment is particularly effective in cases such as theexamples shown in FIGS. 10 and 11.

Seventh Embodiment

A seventh embodiment of the present invention will be described withreference to the drawings.

FIG. 28 is a block diagram showing a configuration of an objectidentification apparatus according to the seventh embodiment of thepresent invention. As shown in FIG. 28, the object identificationapparatus according to the seventh embodiment includes a localdescriptor extracting unit 11, a local descriptor matching unit 16, aninput image difference area determining unit 13, an input imagedifference area descriptor extracting unit 14, a descriptor matchingunit 15, a difference area estimating unit 23, and a difference areadescriptor extracting unit 18. As described above, the objectidentification apparatus according to the seventh embodiment isconfigured so as to combine the object identification apparatusaccording to the second embodiment and the object identificationapparatus according to the sixth embodiment. The object identificationapparatus according to the seventh embodiment differs from the objectidentification apparatus according to the second embodiment in that thedifference area estimating unit 17 has been changed to the differencearea estimating unit 23. Details of the difference area estimating unit23 will be described later. Since other components are similar to thoseof the second embodiment, the components will be denoted by samereference symbols and a detailed description thereof will be omitted.

FIGS. 29, 30, and 31 are block diagrams representing configurationexamples of the difference area estimating unit 23. Each drawing will bedescribed below.

The difference area estimating unit 23 shown in FIG. 29 includes anerroneously-corresponding feature point concentration searching unit2103 and a template matching unit 2301. The erroneously-correspondingfeature point concentration searching unit 2103 shown in FIG. 29 is thesame as the erroneously-corresponding feature point concentrationsearching unit 2103 that is a component of the difference areaestimating unit 21 shown in FIG. 23 and a detailed description thereofwill be omitted.

In addition, the template matching unit 2301 shown in FIG. 29 is similarto the template matching unit 2202 that is a component of the differencearea estimating unit 22 shown in FIG. 27 but differs in that differencecandidate area information is inputted instead of object areainformation. In other words, the template matching unit 2301 shown inFIG. 29 estimates a difference area in a reference image by performingtemplate matching using a template image only on a difference candidatearea in the reference image that is estimated by theerroneously-corresponding feature point concentration searching unit2103. Since the difference area information outputted from the templatematching unit 2301 represents a difference area further narrowed down bythe template matching unit 2301 from difference candidate areasestimated by the erroneously-corresponding feature point concentrationsearching unit 2103, difference area information with high reliabilityis outputted.

Furthermore, the difference area estimating unit 23 shown in FIG. 30 canalso be constituted by a template matching unit 2302 and anerroneously-corresponding feature point concentration searching unit2102. The template matching unit 2302 shown in FIG. 30 is substantiallythe same as the template matching unit 2201 that is a component of thedifference area estimating unit 22 shown in FIG. 26 but differs from thetemplate matching unit 2201 in that the template matching unit 2302outputs difference candidate area information instead of difference areainformation. The difference candidate area information outputted by thetemplate matching unit 2302 may be the same as difference areainformation outputted by the template matching unit 2201 or areainformation representing an area slightly larger than the differencearea information may be adopted instead.

In addition, the erroneously-corresponding feature point concentrationsearching unit 2102 shown in FIG. 30 is the same as theerroneously-corresponding feature point concentration searching unit2102 that is a component of the difference area estimating unit 21 shownin FIG. 22 and a detailed description thereof will be omitted.Specifically, the erroneously-corresponding feature point concentrationsearching unit 2102 shown in FIG. 30 focuses only on points existinginside a difference candidate area in the reference image that isestimated by the template matching unit 2302 among corresponding featurepoints, searches for an area with a concentration oferroneously-corresponding feature points, and estimates a differencearea. Since the difference area information outputted from theerroneously-corresponding feature point concentration searching unit2102 represents a difference area further narrowed down by theerroneously-corresponding feature point concentration searching unit2102 from difference candidate areas estimated by the template matchingunit 2302, difference area information with high reliability isoutputted.

Furthermore, the difference area estimating unit 23 shown in FIG. 31includes an erroneously-corresponding feature point concentrationsearching unit 2103, a template matching unit 2302, and a differencecandidate area overlap detecting unit 2106. Theerroneously-corresponding feature point concentration searching unit2103 shown in FIG. 31 is the same as the erroneously-correspondingfeature point concentration searching unit 2103 that is a component ofthe difference area estimating unit 21 shown in FIG. 23 and a detaileddescription thereof will be omitted. The template matching unit 2302shown in FIG. 31 is the same as the template matching unit 2302 that isa component of the difference area estimating unit 23 shown in FIG. 30and a detailed description thereof will be omitted. The differencecandidate area overlap detecting unit 2106 is the same as the differencecandidate area overlap detecting unit 2106 that is a component of thedifference area estimating unit 21 shown in FIG. 24 and a detaileddescription thereof will be omitted.

Since the difference area information outputted from the differencecandidate area overlap detecting unit 2106 in the configuration shown inFIG. 31 is information regarding an area determined to be a differencecandidate area by both the erroneously-corresponding feature pointconcentration searching unit 2103 and the template matching unit 2302,highly reliable difference area information is outputted.

In the present embodiment, while a difference area in a reference imageneed not be registered in advance in a database in a similar manner tothe second to sixth embodiments, if an image pattern typically observedin the difference area is known in advance in a similar manner to thesixth embodiment, the difference area can be estimated by using theimage pattern as a template image. For example, when exclusivelyidentifying specific mail from among a plurality of mail images whichshow a same envelope and only differ from one another in addresses, anarea where the address is described can be defined as an image patternin which a layout of character strings representing a postal code, anaddress, an addressee, and the like is more or less determined.Therefore, in such cases, the present embodiment is effective. Inaddition, while a difference area in a reference image can be estimatedaccording to the present embodiment regardless of whether the differencearea is an entire object or a part of the object, since a highlyreliable difference area can be obtained in a similar manner to thefifth embodiment as compared to the second embodiment and the like,highly accurate identification can be achieved. Moreover, while thepresent embodiment described heretofore represents a configurationcombining the object identification apparatus according to the secondembodiment with the object identification apparatus according to thesixth embodiment, the configuration shown in FIG. 28 may be consideredas a configuration combining the object identification apparatusaccording to the third embodiment with the object identificationapparatus according to the sixth embodiment. Specifically, when thedifference area estimating unit 23 is configured as shown in FIGS. 29,30, and 31, an object area estimating unit can be added in front of theerroneously-corresponding feature point concentration searching unit2103 and the template matching unit 2302. In this case, since adifference area is to be estimated from among an object area afterremoving the influence of a background, the configuration isparticularly effective in cases such as the examples shown in FIGS. 10and 11.

Eighth Embodiment

An eighth embodiment of the present invention will be described withreference to the drawings.

FIG. 32 is a block diagram showing a configuration of an objectidentification apparatus according to the eighth embodiment of thepresent invention. As shown in FIG. 32, the object identificationapparatus according to the eighth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 12, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 24, and a difference area descriptorextracting unit 18. As described above, the object identificationapparatus according to the eighth embodiment is configured so as tocombine the object identification apparatus according to the fourthembodiment with the object identification apparatus according to thesixth embodiment. The object identification apparatus according to theeighth embodiment differs from the object identification apparatusaccording to the fourth embodiment in that the difference areaestimating unit 20 has been changed to the difference area estimatingunit 24. Details of the difference area estimating unit 24 will bedescribed later. Since other components are similar to those of thefourth embodiment, the components will be denoted by same referencesymbols and a detailed description thereof will be omitted.

FIGS. 33 to 35 are block diagrams representing configuration examples ofthe difference area estimating unit 24. Each drawing will be describedbelow.

The difference area estimating unit 24 shown in FIG. 33 includes aconverted image generating unit 2001, a difference image generating unit2002, an object area estimating unit 2003, a large difference areadetecting unit 2101, and a template matching unit 2301. The convertedimage generating unit 2001, the difference image generating unit 2002,and the object area estimating unit 2003 shown in FIG. 33 are the sameas the converted image generating unit 2001, the difference imagegenerating unit 2002, and the object area estimating unit 2003 that arecomponents of the difference area estimating unit 20 shown in FIG. 19and a detailed description thereof will be omitted. The large differencearea detecting unit 2101 shown in FIG. 33 is the same as the largedifference area detecting unit 2101 that is a component of thedifference area estimating unit 21 shown in FIG. 22 and a detaileddescription thereof will be omitted. In addition, the template matchingunit 2301 shown in FIG. 33 is the same as the template matching unit2301 that is a component of the difference area estimating unit 23 shownin FIG. 29 and a detailed description thereof will be omitted.

In other words, the template matching unit 2301 shown in FIG. 33estimates a difference area in a reference image by performing templatematching using a template image only on a difference candidate area inthe reference image that is estimated by the large difference areadetecting unit 2101. Since the difference area information outputtedfrom the template matching unit 2301 represents a difference areafurther narrowed down by the template matching unit 2301 from differencecandidate areas estimated by the large difference area detecting unit2101, difference area information with high reliability is outputted.

Furthermore, the difference area estimating unit 24 shown in FIG. 34includes a converted image generating unit 2001, a template matchingunit 2302, a difference image generating unit 2104, and a largedifference area detecting unit 2105. The converted image generating unit2001 shown in FIG. 34 is the same as the converted image generating unit2001 that is a component of the difference area estimating unit 20 shownin FIG. 19 and a detailed description thereof will be omitted. Inaddition, the template matching unit 2302 shown in FIG. 34 is the sameas the template matching unit 2302 that is a component of the differencearea estimating unit 23 shown in FIG. 30 and a detailed descriptionthereof will be omitted. The difference image generating unit 2104 andthe large difference area detecting unit 2105 shown in FIG. 34 arerespectively the same as the difference image generating unit 2104 andthe large difference area detecting unit 2105 that are components of thedifference area estimating unit 21 shown in FIG. 23 and a detaileddescription thereof will be omitted.

Since the difference area information outputted from the largedifference area detecting unit 2105 represents a difference area furthernarrowed down by the large difference area detecting unit 2105 fromdifference candidate areas estimated by the template matching unit 2302,difference area information with high reliability is outputted.

In addition, the difference area estimating unit 24 shown in FIG. 35includes a converted image generating unit 2001, a difference imagegenerating unit 2002, an object area estimating unit 2003, a largedifference area detecting unit 2101, a template matching unit 2302, anda difference candidate area overlap detecting unit 2106.

The converted image generating unit 2001, the difference imagegenerating unit 2002, and the object area estimating unit 2003 shown inFIG. 35 are the same as the converted image generating unit 2001, thedifference image generating unit 2002, and the object area estimatingunit 2003 that are components of the difference area estimating unit 20shown in FIG. 19 and a detailed description thereof will be omitted. Inaddition, the large difference area detecting unit 2101 shown in FIG. 35is the same as the large difference area detecting unit 2101 that is acomponent of the difference area estimating unit 21 shown in FIG. 22 anda detailed description thereof will be omitted. Furthermore, thetemplate matching unit 2302 shown in FIG. 35 is the same as the templatematching unit 2302 that is a component of the difference area estimatingunit 23 shown in FIG. 30 and a detailed description thereof will beomitted. Moreover, the difference candidate area overlap detecting unit2106 shown in FIG. 35 is the same as the difference candidate areaoverlap detecting unit 2106 that is a component of the difference areaestimating unit 21 shown in FIG. 24 and a detailed description thereofwill be omitted.

Since the difference area information outputted from the differencecandidate area overlap detecting unit 2106 in the configuration shown inFIG. 35 is information regarding an area determined to be a differencecandidate area by both the large difference area detecting unit 2101 andthe template matching unit 2302, highly reliable difference areainformation is outputted.

In the present embodiment, a difference area in a reference image neednot be registered in advance in a database in a similar manner to thesecond to seventh embodiments, and if an image pattern typicallyobserved in the difference area is known in advance in a similar mannerto the sixth and seventh embodiments, the difference area can beestimated by using the image pattern as a template image. For example,when exclusively identifying specific mail from among a plurality ofmail images which show a same envelope and only differ from one anotherin addresses, an area where the address is described can be defined asan image pattern in which a layout of character strings representing apostal code, an address, an addressee, and the like is more or lessdetermined. Therefore, in such cases, the present embodiment iseffective. In addition, while a difference area in a reference image canbe estimated according to the present embodiment regardless of whetherthe difference area is an entire object or a part of the object, since ahighly reliable difference area can be obtained in a similar manner tothe fifth and seventh embodiments as compared to the second embodimentand the like, highly accurate identification can be achieved. Moreover,in the present embodiment, when the difference area estimating unit 24is configured as shown in FIG. 34 or 35, an object area estimating unitcan be added in front of the template matching unit 2302. In this case,since a difference area is to be estimated from among an object areaafter removing the influence of a background, the configuration isparticularly effective in the cases shown in FIGS. 10 and 11.

Ninth Embodiment

A ninth embodiment of the present invention will be described withreference to the drawings.

FIG. 36 is a block diagram showing a configuration of an objectidentification apparatus according to the ninth embodiment of thepresent invention. As shown in FIG. 36, the object identificationapparatus according to the ninth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 16, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 14, a descriptor matching unit 15, adifference area estimating unit 25, and a difference area descriptorextracting unit 18.

As described above, the object identification apparatus according to theninth embodiment is configured so as to combine the objectidentification apparatus according to the fifth embodiment and theobject identification apparatus according to the sixth embodiment. Theobject identification apparatus according to the ninth embodimentdiffers from the object identification apparatus according to the fifthembodiment in that the difference area estimating unit 21 has beenchanged to the difference area estimating unit 25. Details of thedifference area estimating unit 25 will be described later. Since othercomponents are similar to those of the fifth embodiment, the componentswill be denoted by same reference symbols and a detailed descriptionthereof will be omitted.

The difference area estimating unit 25 can be configured as acombination of a configuration in which a difference area is estimatedsolely by the erroneously-corresponding feature point concentrationsearching unit 701 as shown in FIG. 14, a configuration in which adifference area is estimated by the converted image generating unit2001, the difference image generating unit 2002, the object areaestimating unit 2003, and the large difference area detecting unit 2004as shown in FIG. 19, and a configuration in which a difference area isestimated solely by the template matching unit 2201 as shown in FIG. 26.When the three configurations for estimating a difference area areconfigured so that, after difference candidate areas are estimated byone configuration, the difference candidate areas are further narroweddown by another configuration, the three configurations for estimating adifference area may be configured in any order. Alternatively, the threeconfigurations for estimating a difference area may be configured sothat a difference candidate area is estimated by each configuration andan overlapping area of the difference candidate areas estimated by therespective configurations is outputted as a final difference area.Alternatively, the three configurations for estimating a difference areamay be configured so that difference candidate areas are first estimatedby one configuration, the difference candidate areas are respectivelynarrowed down by the two remaining configurations, and an overlappingarea of the difference candidate areas estimated by the twoconfigurations is outputted as a final difference area. Alternatively,the three configurations for estimating a difference area may beconfigured so that difference candidate areas are first estimated by twoconfigurations, a difference candidate area is narrowed down by the oneremaining configuration from overlapping areas of the differencecandidate areas estimated by the two configurations, and a narrowed-downarea is outputted as a final difference area.

FIGS. 37 and 38 are block diagrams representing configuration examplesof the difference area estimating unit 25. Each drawing will bedescribed below.

As shown in FIG. 37, the difference area estimating unit 25 includes aconverted image generating unit 2001, a difference image generating unit2002, an object area estimating unit 2003, a large difference areadetecting unit 2101, a template matching unit 2501, and anerroneously-corresponding feature point concentration searching unit2102.

The converted image generating unit 2001, the difference imagegenerating unit 2002, and the object area estimating unit 2003 shown inFIG. 37 are the same as the converted image generating unit 2001, thedifference image generating unit 2002, and the object area estimatingunit 2003 that are components of the difference area estimating unit 20shown in FIG. 19 and a detailed description thereof will be omitted. Thelarge difference area detecting unit 2101 shown in FIG. 37 is the sameas the large difference area detecting unit 2101 shown in FIG. 22 and adetailed description thereof will be omitted.

The template matching unit 2501 shown in FIG. 37 is substantially thesame as the template matching unit 2301 that is a component of thedifference area estimating unit 23 shown in FIG. 29 but differs in thatthe template matching unit 2501 outputs difference candidate areainformation instead of difference area information.

The erroneously-corresponding feature point concentration searching unit2102 shown in FIG. 37 is the same as the erroneously-correspondingfeature point concentration searching unit 2102 that is a component ofthe difference area estimating unit 21 shown in FIG. 22 and a detaileddescription thereof will be omitted. Specifically, theerroneously-corresponding feature point concentration searching unit2102 shown in FIG. 37 focuses only on points existing inside adifference candidate area in the reference image that is estimated bythe template matching unit 2501 among corresponding feature points,searches for an area with a concentration of erroneously-correspondingfeature points, and estimates a difference area. In addition, thetemplate matching unit 2501 similarly estimates a difference areacandidate in a reference image by performing template matching using atemplate image only on a difference candidate area in the referenceimage that is estimated by the large difference area detecting unit2101. Since the difference area information outputted from theerroneously-corresponding feature point concentration searching unit2102 represents a difference area further narrowed down by theerroneously-corresponding feature point concentration searching unit2102 from difference candidate areas estimated by the large differencearea detecting unit 2101 and the template matching unit 2501, differencearea information with high reliability is outputted.

In addition, as shown in FIG. 38, the difference area estimating unit 25can also be constituted by a converted image generating unit 2001, adifference image generating unit 2002, an object area estimating unit2003, a large difference area detecting unit 2101, a template matchingunit 2302, an erroneously-corresponding feature point concentrationsearching unit 2103, and a difference candidate area overlap detectingunit 2502. The converted image generating unit 2001, the differenceimage generating unit 2002, and the object area estimating unit 2003shown in FIG. 38 are the same as the converted image generating unit2001, the difference image generating unit 2002, and the object areaestimating unit 2003 that are components of the difference areaestimating unit 20 shown in FIG. 19 and a detailed description thereofwill be omitted. In addition, the large difference area detecting unit2101 shown in FIG. 38 is the same as the large difference area detectingunit 2101 that is a component of the difference area estimating unit 21shown in FIG. 22 and a detailed description thereof will be omitted.Furthermore, the template matching unit 2302 shown in FIG. 38 is thesame as the template matching unit 2302 that is a component of thedifference area estimating unit 23 shown in FIG. 30 and a detaileddescription thereof will be omitted. Moreover, theerroneously-corresponding feature point concentration searching unit2103 shown in FIG. 38 is the same as the erroneously-correspondingfeature point concentration searching unit 2103 that is a component ofthe difference area estimating unit 21 shown in FIG. 23 and a detaileddescription thereof will be omitted.

The difference candidate area overlap detecting unit 2502 shown in FIG.38 is similar to the difference candidate area overlap detecting unit2106 that is a component of the difference area estimating unit 21 shownin FIG. 24. The difference candidate area overlap detecting unit 2502shown in FIG. 38 differs from the difference candidate area overlapdetecting unit 2106 in that the difference candidate area overlapdetecting unit 2502 receives three pieces of difference candidate areainformation respectively outputted from the large difference areadetecting unit 2101, the template matching unit 2302, and theerroneously-corresponding feature point concentration searching unit2103, determines that an overlapping area of the three differencecandidate areas to be a difference area, and outputs difference areainformation thereof. Since the difference area information outputtedfrom the difference candidate area overlap detecting unit 2502 isinformation regarding an area determined to be a difference candidatearea by all of the large difference area detecting unit 2101, thetemplate matching unit 2302, and the erroneously-corresponding featurepoint concentration searching unit 2103, highly reliable difference areainformation is outputted.

Moreover, the difference area estimating unit 25 can adopt aconfiguration other than those shown in FIGS. 37 and 38. For example,with respect to difference candidate areas first estimated by theerroneously-corresponding feature point concentration searching unit2103, the template matching unit 2501 may further narrow down thedifference candidate areas. Final difference area information may befurther estimated from among the narrowed down difference candidateareas by a combination of the converted image generating unit 2001, thedifference image generating unit 2104, and the large difference areadetecting unit 2105. In addition, the processing order of estimation ofa difference candidate area using an erroneously-corresponding featurepoint concentration search, estimation of a difference candidate areausing template matching, and estimation of a difference candidate areausing a large difference area detection may differ from the orderfollowed in the configuration examples described heretofore.

In the present embodiment, while a difference area in a reference imageneed not be registered in advance in a database in a similar manner tothe second to eighth embodiments, if an image pattern typically observedin the difference area is known in advance in a similar manner to thesixth to eighth embodiments, the difference area can be estimated byusing the image pattern as a template image. For example, whenexclusively identifying specific mail from among a plurality of mailimages which show a same envelope and only differ from one another inaddresses, an area where the address is described can be defined as animage pattern in which a layout of character strings representing apostal code, an address, an addressee, and the like is more or lessdetermined. Therefore, in such cases, the present embodiment iseffective. In addition, while a difference area in a reference image canbe estimated according to the present embodiment regardless of whetherthe difference area is an entire object or a part of the object, since ahighly reliable difference area can be obtained in a similar manner tothe fifth, seventh, and eighth embodiments as compared to the secondembodiment and the like, highly accurate identification can be achieved.Moreover, in the present embodiment, when the difference area estimatingunit 25 is configured as shown in FIG. 38, an object area estimatingunit can be added in front of the template matching unit 2302 and theerroneously-corresponding feature point concentration searching unit2103. In this case, since a difference area is to be estimated fromamong an object area after removing the influence of a background, theconfiguration is particularly effective in the cases shown in FIGS. 10and 11.

Tenth Embodiment

A tenth embodiment of the present invention will be described withreference to the drawings.

FIG. 39 is a block diagram showing a configuration example of an objectidentification apparatus according to the tenth embodiment of thepresent invention. As shown in FIG. 39, the object identificationapparatus according to the tenth embodiment includes a local descriptorextracting unit 11, a local descriptor matching unit 12, an input imagedifference area determining unit 13, an input image difference areadescriptor extracting unit 26, and a descriptor matching unit 27.

As described above, the object identification apparatus according to thetenth embodiment differs from the object identification apparatusaccording to the first embodiment in that the input image differencearea descriptor extracting unit 14 and the descriptor matching unit 15have been changed to the input image difference area descriptorextracting unit 26 and the descriptor matching unit 27. Details of theinput image difference area descriptor extracting unit 26 and thedescriptor matching unit 27 will be described later. Since othercomponents are similar to those of the first embodiment, the componentswill be denoted by same reference symbols and a detailed descriptionthereof will be omitted.

FIG. 40 is a block diagram representing a configuration example of theinput image difference area descriptor extracting unit 26. As shown inFIG. 40, the input image difference area descriptor extracting unit 26includes a difference area local descriptor extracting unit 2601.

The difference area local descriptor extracting unit 2601 receives inputimage difference area information outputted from the input imagedifference area determining unit 13 and a local descriptor 1 that is alocal descriptor extracted from an input image by the local descriptorextracting unit 11, and extracts a descriptor 1 based on the receivedinput image difference area information and the local descriptor. As thedescriptor 1 extracted at this point, a feature point existing in adifference area in the input image is searched based on coordinateinformation of a feature point that forms a basis when describing thelocal descriptor 1 and a descriptor describing information in aperiphery of the feature point as a local descriptor is outputted as thedescriptor 1. In other words, the descriptor 1 extracted at this pointis a descriptor generated by cutting out a part of the local descriptor1. In the present embodiment, a descriptor 2 to be matched with thedescriptor 1 by the descriptor matching unit 27 is a descriptorgenerated in a similar manner to the descriptor 1 by cutting out a partof a local descriptor 2 that is a local descriptor extracted from areference image.

FIG. 41 is a block diagram representing a configuration example of thedescriptor matching unit 27. As shown in FIG. 41, the descriptormatching unit 27 includes a corresponding feature point determining unit201, an erroneous corresponding point removing unit 2701, anidentification score calculating unit 203, and a threshold determiningunit 2702. The corresponding feature point determining unit 201 and theidentification score calculating unit 203 shown in FIG. 41 are the sameas the corresponding feature point determining unit 201 and theidentification score calculating unit 203 that are components of thelocal descriptor matching unit 12 shown in FIG. 4 and a detaileddescription thereof will be omitted.

The erroneous corresponding point removing unit 2701 shown in FIG. 41 issubstantially the same as the erroneous corresponding point removingunit 202 that is a component of the local descriptor matching unit 12shown in FIG. 4 but differs therefrom in that the erroneouscorresponding point removing unit 2701 does not output geometrictransformation information and exclusively outputs corresponding featurepoint information. The threshold determining unit 2702 shown in FIG. 41is substantially the same as the threshold determining unit 204 that isa component of the local descriptor matching unit 12 shown in FIG. 4 butdiffers therefrom in that the threshold determining unit 2702 outputs adifference area identified image ID instead of a local featureidentified image ID.

As described above, in the present embodiment, a descriptor of the inputimage and a descriptor of the reference image that are used in thematching by the descriptor matching unit 27 are respectively parts of adescriptor of the input image and a descriptor of the reference imagethat are used in the matching (determination) by the local descriptormatching unit 12.

Unlike the first to ninth embodiments, in the present embodiment, adescriptor generated by cutting out a part of a local descriptor is usedto identify a difference area. Therefore, when generating a descriptorwith the input image difference area descriptor extracting unit, a localdescriptor extracted from an input image need only be inputted and theinput image itself is not required. As a result, when an objectidentification apparatus is configured as a server-client system inwhich only extraction of a local descriptor is performed by a client andall other processes are performed by a server, according to the presentembodiment, only a local descriptor that is lighter in weight than aninput image need be transmitted to the server and a processing timeuntil obtaining an identification result can be reduced. In addition,while the descriptor matching unit according to the present embodimentperforms substantially the same process as the local descriptor matchingunit, since the descriptor matching unit removes influence due tocorrespondence of feature points detected outside a difference area anda matching limited to the difference area can be performed, a differenceinside an object can be distinguished more clearly and, as a result,identification with higher accuracy can be achieved as compared toconventional systems that use an entire local descriptor extracted froman entire image. Moreover, while FIG. 39 described heretofore in anorderly manner as a configuration example of the present embodiment is aconfiguration based on the first embodiment, a configurationrespectively based on the second to ninth embodiments can be similarlyadopted. In other words, a configuration can be adopted in which thelocal descriptor 1 is inputted instead of an input image to the inputimage difference area descriptor extracting unit 14 in the configurationexamples according to the second to ninth embodiments.

Eleventh Embodiment

An eleventh embodiment of the present invention will be described withreference to the drawings.

FIG. 42 is a block diagram showing a configuration example of an objectidentification apparatus according to the eleventh embodiment of thepresent invention. As shown in FIG. 42, the object identificationapparatus according to the eleventh embodiment includes a localdescriptor extracting unit 11, a local descriptor matching unit 28, aninput image difference area determining unit 13, an input imagedifference area descriptor extracting unit 14, a descriptor matchingunit 29, and an identification score integration determining unit 30.

As described above, the object identification apparatus according to theeleventh embodiment differs from the first embodiment in that the localdescriptor matching unit 12 and the descriptor matching unit 15 of theobject identification apparatus according to the first embodiment havebeen changed to the local descriptor matching unit 28 and the descriptormatching unit 29 and that the identification score integrationdetermining unit 30 has been added as a new component. Details of thelocal descriptor matching unit 28, the descriptor matching unit 29, andthe identification score integration determining unit 30 will bedescribed later. Since other components are similar to those of thefirst embodiment, the components will be denoted by same referencesymbols and a detailed description thereof will be omitted.

FIG. 43 is a block diagram representing a configuration example of thelocal descriptor matching unit 28. As shown in FIG. 43, the localdescriptor matching unit 28 includes a corresponding feature pointdetermining unit 201, an erroneous corresponding point removing unit202, an identification score calculating unit 203, and a thresholddetermining unit 2801.

The corresponding feature point determining unit 201, the erroneouscorresponding point removing unit 202, and the identification scorecalculating unit 203 shown in FIG. 43 are the same as the correspondingfeature point determining unit 201, the erroneous corresponding pointremoving unit 202, and the identification score calculating unit 203that are components of the local descriptor matching unit 12 shown inFIG. 4 and a detailed description thereof will be omitted.

The threshold determining unit 2801 shown in FIG. 43 is substantiallythe same as the threshold determining unit 204 that is a component ofthe local descriptor matching unit 12 shown in FIG. 4 but differs inthat the threshold determining unit 2801 outputs not only a localfeature identified image ID but also an identification score with alocal feature extracted from a reference image corresponding to thelocal feature identified image ID (or a reference image group associatedwith the local feature identified image ID). A threshold set by thethreshold determining unit 2801 may be set laxer than the threshold setby the threshold determining unit 204 so that a large number of localfeature identified image IDs and identification scores are outputted.

FIG. 44 is a block diagram representing a configuration example of thedescriptor matching unit 29. As shown in FIG. 44, the descriptormatching unit 29 includes a difference area identification scorecalculating unit 501 and a threshold determining unit 2901. Thedifference area identification score calculating unit 501 shown in FIG.44 is the same as the difference area identification score calculatingunit 501 that is a component of the descriptor matching unit 15 shown inFIG. 6 and a detailed description thereof will be omitted.

The threshold determining unit 2901 shown in FIG. 44 is substantiallythe same as the threshold determining unit 502 that is a component ofthe descriptor matching unit 15 shown in FIG. 6 but differs in that thethreshold determining unit 2901 outputs not only a difference areaidentified image ID but also a difference area identification score witha local feature extracted from a difference area of a reference imagecorresponding to the difference area identified image ID or a referenceimage group associated with the difference area identified image ID. Athreshold set by the threshold determining unit 2901 may be set laxerthan the threshold set by the threshold determining unit 502 so that alarge number of difference area identified image IDs and difference areaidentification scores are outputted.

FIG. 45 is a block diagram representing a configuration example of theidentification score integration determining unit 30. As shown in FIG.45, the identification score integration determining unit 30 includes anidentification score integrating unit 3001 and a threshold determiningunit 3002.

The identification score integrating unit 3001 receives anidentification score outputted from the local descriptor matching unit28 and a difference area identification score outputted from thedescriptor matching unit 29 and calculates and outputs an integratedscore based on the received scores. At this point, for example, aproduct of an identification score and a difference area identificationscore corresponding to a same image ID may be calculated and a valuethereof may be outputted as an integrated score.

The threshold determining unit 3002 shown in FIG. 45 is substantiallythe same as the threshold determining unit 204 that is a component ofthe local descriptor matching unit 12 shown in FIG. 4 or the thresholddetermining unit 502 that is a component of the local descriptormatching unit 15 shown in FIG. 6. The threshold determining unit 3002differs from the threshold determining unit 204 and the thresholddetermining unit 502 in that the threshold determining unit 3002compares the integrated score outputted from the identification scoreintegrating unit 3001 with a prescribed threshold, determines that theinput image and the reference image are images that share a same objectas subjects when the integrated score is equal to or higher than thethreshold and outputs an image ID of the input image as an identifiedimage ID, and determines that the input image and the reference imageare not images that share a same object as subjects when the integratedscore is lower than the threshold. In other words, the thresholddetermining unit 3002 determines whether or not the input image and thereference image are images that share a same object as subjects based ona result of matching by the local descriptor matching unit 28 and aresult of matching by the descriptor matching unit 29.

Unlike the first to tenth embodiments, in the present embodiment, afinal identification result is not determined solely based on adifference area identification score but is determined based on a scorethat integrates the difference area identification score with anidentification score from a local descriptor. When images showing a sameobject are photographed in an adverse environment (for example, a darkenvironment) and other similar objects are photographed in an idealenvironment, if the similar objects are similar not only in texture butalso in color, a correct identification cannot be performed by onlyusing a descriptor extracted from a difference area. However, bycombining with a result of an identification based on a localdescriptor, an identification result with respect to a same object canbe relatively improved. Moreover, while FIG. 42 described heretofore inan orderly manner as a configuration example of the present embodimentis a configuration based on the first embodiment, a configurationrespectively based on the second to tenth embodiments can be similarlyadopted. In other words, in the configuration examples of the second totenth embodiments, configurations can be adopted in which anidentification score is outputted from a local descriptor matching unit,a difference area identification score is outputted from a descriptormatching unit, and both the identification score and the difference areaidentification score are inputted to an identification score integrationdetermining unit.

It should be noted that specific configurations of the present inventionare not limited to those of the embodiments described above and thatvarious modifications and changes made to the present invention withoutdeparting from the spirit and scope thereof are to be included in thepresent invention.

The present application claims priority on the basis of Japanese PatentApplication No. 2012-184534 filed on Aug. 23, 2012, the entire contentsof which are incorporated herein by reference.

While the present invention has been described with reference toembodiments, the present invention is not intended to be limited to theembodiments described above. Various modifications to configurations anddetails of the present invention will occur to those skilled in the artwithout departing from the scope of the present invention.

A part of or all of the embodiments above may also be described as, butnot limited to, the appendices provided below.

APPENDIX 1

An object identification apparatus including:

a local descriptor matching unit for determining whether or notrespective descriptors of feature points extracted from an input imageand respective descriptors of feature points extracted from a referenceimage correctly correspond to one another;

an input image difference area descriptor extracting unit for extractinga descriptor of an area in the input image corresponding to a positionof an image area obtained by performing a geometric transformation forcorrecting a geometric deviation between the input image and thereference image on a prescribed area of the reference image when a scorebased on the number of combinations of descriptors determined tocorrespond correctly by the local descriptor matching unit is equal toor larger than a prescribed value; and

a descriptor matching unit for matching the descriptor extracted by theinput image difference area descriptor extracting unit with a descriptorextracted from the prescribed area of the reference image, and foroutputting a matching result.

APPENDIX 2

The object identification apparatus according to Appendix 1, furtherincluding a storage unit for storing information regarding theprescribed area of the reference image.

APPENDIX 3

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area including a featurepoint in the reference image, to which a descriptor is determined tocorrespond erroneously by the local descriptor matching unit.

APPENDIX 4

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area including a featurepoint in the reference image, to which a descriptor is determined tocorrespond erroneously by the local descriptor matching unit, in an areashowing an object.

APPENDIX 5

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image includes an area in which adifference between an image obtained by performing a geometrictransformation for correcting a geometric deviation between the inputimage and the reference image on the input image and the reference imageis equal to or greater than a prescribed value.

APPENDIX 6

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area which includes afeature point, to which a descriptor is determined to corresponderroneously by the local descriptor matching unit, in an area showing anobject, and which includes an area in which a difference between animage obtained by performing a geometric transformation for correcting ageometric deviation between the input image and the reference image onthe input image and the reference image is equal to or greater than aprescribed value.

APPENDIX 7

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area including an areawhose degree of similarity with a prescribed pattern image is equal toor greater than a prescribed value in the reference image.

APPENDIX 8

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area which includes anarea, to which a descriptor is determined to correspond erroneously bythe local descriptor matching unit and whose degree of similarity with aprescribed pattern image is equal to or greater than a prescribed valuein the reference image.

APPENDIX 9

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area including an area inwhich a difference between an image obtained by performing a geometrictransformation for correcting a geometric deviation between the inputimage and the reference image on the input image and the reference imageis equal to or greater than a prescribed value and whose degree ofsimilarity with a prescribed pattern image is equal to or greater than aprescribed value in the reference image.

APPENDIX 10

The object identification apparatus according to Appendix 1, wherein theprescribed area of the reference image is an area which includes afeature point to which a descriptor is determined to corresponderroneously by the local descriptor matching unit, and which includes anarea in which a difference between an image obtained by performing ageometric transformation for correcting a geometric deviation betweenthe input image and the reference image on the input image and thereference image is equal to or greater than a prescribed value, andmoreover whose degree of similarity with a prescribed pattern image isequal to or greater than a prescribed value in the reference image.

APPENDIX 11

The object identification apparatus according to any one of Appendices 1to 10, wherein a descriptor of the input image and a descriptor of thereference image that are used in a matching by the descriptor matchingunit are respectively parts of the descriptor of the input image and thedescriptor of the reference image that have been used in thedetermination by the local descriptor matching unit.

APPENDIX 12

The object identification apparatus according to any one of Appendices 1to 11, further including an integration determining unit for determiningwhether or not the input image and the reference image are imagessharing a same object as subjects based on a result of the determinationby the local descriptor matching unit and a result of the matching bythe descriptor matching unit.

APPENDIX 13

An object identification method including:

a local descriptor matching step of determining whether or notrespective descriptors of feature points extracted from an input imageand respective descriptors of feature points extracted from a referenceimage correctly correspond to one another;

an input image difference area descriptor extracting step of extractinga descriptor of an area in the input image corresponding to a positionof an image area obtained by performing a geometric transformation forcorrecting a geometric deviation between the input image and thereference image on a prescribed area of the reference image when a scorebased on the number of combinations of descriptors determined tocorrespond correctly in the determining step is equal to or larger thana prescribed value; and

a descriptor matching step of matching the descriptor extracted in theextracting step with a descriptor extracted from the prescribed area ofthe reference image, and outputting a matching result.

APPENDIX 14

A program causing a computer to function as:

a local descriptor matching unit for determining whether or notrespective descriptors of feature points extracted from an input imageand respective descriptors of feature points extracted from a referenceimage correctly correspond to one another;

an input image difference area descriptor extracting unit for extractinga descriptor of an area in the input image corresponding to a positionof an image area obtained by performing a geometric transformation forcorrecting a geometric deviation between the input image and thereference image on a prescribed area of the reference image when a scorebased on the number of combinations of descriptors determined tocorrespond correctly by the local descriptor matching unit is equal toor larger than a prescribed value; and

a descriptor matching unit for matching the descriptor extracted by theinput image difference area descriptor extracting unit with a descriptorextracted from the prescribed area of the reference image, and foroutputting a matching result.

With conventional object identification that only uses a localdescriptor, it is difficult to accurately identify products which belongto a same brand and which only differ from each other in colors ofpackages or a part of characters and accurately identify mail whichshare a same envelope and only differ from one another in addresses.However, according to the present invention, fine differences thatcannot be identified by conventional matching using only a localdescriptor can now be distinguished and images showing a same object canbe exclusively identified. Accordingly, the present invention can beapplied to a barcode-less POS system, an article inspection system, anautomatic mail sorting system, and the like.

-   11 local descriptor extracting unit-   12 local descriptor matching unit-   13 input image difference area determining unit-   14 input image difference area descriptor extracting unit-   15 descriptor matching unit-   16 local descriptor matching unit-   17 difference area estimating unit-   18 difference area descriptor extracting unit-   19, 20, 21, 22, 23, 24, 25 difference area estimating unit-   26 input image difference area descriptor extracting unit-   27 descriptor matching unit-   28 local descriptor matching unit-   29 descriptor matching unit-   30 identification score integration determining unit-   101 brightness information extracting unit-   102 local feature point detecting unit-   103 local descriptor generating unit-   201 corresponding feature point determining unit-   202 erroneous corresponding point removing unit-   203 identification score calculating unit-   204 threshold determining unit-   401 difference area image generating unit-   402 difference area descriptor calculating unit-   501 difference area identification score calculating unit-   502 threshold determining unit-   701 erroneously-corresponding feature point concentration searching    unit-   801 difference area image generating unit-   802 difference area descriptor calculating unit-   901 object area estimating unit-   902 erroneously-corresponding feature point concentration searching    unit-   2001 converted image generating unit-   2002 difference image generating unit-   2003 object area estimating unit-   2004, 2101 large difference area detecting unit-   2102, 2103 erroneously-corresponding feature point concentration    searching unit-   2104 difference image generating unit-   2105 large difference area detecting unit-   2106 difference candidate area overlap detecting unit-   2201, 2202, 2301, 2302, 2501 template matching unit-   2502 difference candidate area overlap detecting unit-   2601 difference area local descriptor extracting unit-   2701 erroneous corresponding point removing unit-   2702, 2801, 2901, 3002 threshold determining unit-   3001 identification score integrating unit

I claim:
 1. An object identification apparatus comprising: a processor;and a memory coupled to the processor; wherein the memory storesinstructions that cause the processor to: determine whether or notrespective descriptors of feature points extracted from an input imageand respective descriptors of feature points extracted from a referenceimage correctly correspond to one another; extract a first descriptor ofan area in the input image corresponding to a position of an image areaobtained by performing a geometric transformation for correcting ageometric deviation between the input image and the reference image on aprescribed area of the reference image when a score based on the numberof combinations of descriptors determined to correspond correctly isequal to or larger than a prescribed value; match the first descriptorwith a second descriptor extracted from the prescribed area of thereference image; and output a matching result, wherein the firstdescriptor and the second descriptor are respectively parts of thedescriptor of the input image and the descriptor of the reference imagethat have been used in the determination of whether or not thedescriptors of feature points extracted from an input image andrespective descriptors of feature points extracted from a referenceimage correctly correspond to one another.
 2. The object identificationapparatus according to claim 1, wherein information regarding theprescribed area of the reference image is stored.
 3. The objectidentification apparatus according to claim 1, wherein the prescribedarea of the reference image is an area including a feature point in thereference image, to which a descriptor is determined to corresponderroneously.
 4. The object identification apparatus according to claim1, wherein the prescribed area of the reference image is an areaincluding a feature point in the reference image, to which a descriptoris determined to correspond erroneously, in an area showing an object.5. The object identification apparatus according to claim 1, wherein theprescribed area of the reference image includes an area in which adifference between an image obtained by performing a geometrictransformation for correcting a geometric deviation between the inputimage and the reference image on the input image and the reference imageis equal to or greater than a prescribed value.
 6. The objectidentification apparatus according to claim 1, wherein the prescribedarea of the reference image is an area which includes a feature point,to which a descriptor is determined to correspond erroneously, in anarea showing an object, and which includes an area in which a differencebetween an image obtained by performing a geometric transformation forcorrecting a geometric deviation between the input image and thereference image on the input image and the reference image is equal toor greater than a prescribed value.
 7. The object identificationapparatus according to claim 1, wherein the prescribed area of thereference image is an area including an area whose degree of similaritywith a prescribed pattern image is equal to or greater than a prescribedvalue in the reference image.
 8. An object identification methodcomprising: a local descriptor matching step of determining whether ornot respective descriptors of feature points extracted from an inputimage and respective descriptors of feature points extracted from areference image correctly correspond to one another; an input imagedifference area descriptor extracting step of extracting a descriptor ofan area in the input image corresponding to a position of an image areaobtained by performing a geometric transformation for correcting ageometric deviation between the input image and the reference image on aprescribed area of the reference image when a score based on the numberof combinations of descriptors determined to correspond correctly in thedetermining step is equal to or larger than a prescribed value; and adescriptor matching step of matching the descriptor extracted in theextracting step with a descriptor extracted from the prescribed area ofthe reference image, and outputting a matching result, wherein adescriptor of the input image and a descriptor of the reference imagethat are used in a matching in the descriptor matching step arerespectively parts of the descriptor of the input image and thedescriptor of the reference image that have been used in thedetermination in the local descriptor matching step.
 9. A non-transitorycomputer-readable medium that stores a program causing a computer to:determine whether or not respective descriptors of feature pointsextracted from an input image and respective descriptors of featurepoints extracted from a reference image correctly correspond to oneanother; extract a first descriptor of an area in the input imagecorresponding to a position of an image area obtained by performing ageometric transformation for correcting a geometric deviation betweenthe input image and the reference image on a prescribed area of thereference image when a score based on the number of combinations ofdescriptors determined to correspond correctly is equal to or largerthan a prescribed value; match the first descriptor with a seconddescriptor extracted from the prescribed area of the reference image;and output a matching result, wherein the first descriptor and thesecond descriptor are respectively parts of the descriptor of the inputimage and the descriptor of the reference image that have been used inthe determination of whether or not the descriptors of feature pointsextracted from an input image and respective descriptors of featurepoints extracted from a reference image correctly correspond to oneanother.